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Part I 


Iterative Bootstrap Hybrid 
Decoding Using the Multiple 
Stack Algorithm 
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Sequential Decoding 
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long enough to cause intormation loss, parucuid 

close to the channel cut-off rate, R 0 . 



The Stack Algorithm 
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The Multiple Stack Algorithm 

The Multiple Stack Algorithm (MSA) is a sequential 
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corresponding metric. The decoder then returns to work in 

the immediately preceding stack. 






information provided by the extra redundancy and uses it to 
adjust the metric used by the sequential decoder. 
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Part II 


Soft-Output Sequential 
Decoder 
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The Soft-Output Viterbi 
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A Soft-Output Sequential Decoder 
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Part III 


Concatenated Turbo Codes 



Bit Error Rate 


Performance of Turbo Codes 
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E„/N, (dB) 







Turbo codes have an overall distance spectrum 
similar to a random code, and thus they are capable 
of superior performance on noisy channels. With a 
block length of 2 16 , they can achieve a bit error rate 
(BER) of 10' 5 at a channel signal-to-noise ratio (SNR) 
only 0.7 dB away from capacity, an improvement of 
about 2 dB compared to the best previously known 
codes. Furthermore, an iterative MAP decoding 
algorithm allows this remarkable performance to be 
achieved with moderate decoding complexity. 

Problems with Turbo codes: 

-Due to their small free distance, there seems to be no 
way to eliminate the error floor by increasing the 
block size or improving the interleaver. 

-Near capacity performance requires considerable 
decoding effort, e.g., to achieve a BER of 10~ 5 only 0.7 
dB away from capacity, 18 iterations of MAP 
decoding are required. 

-Performance depends on the interleaver size, thus 
resulting in a long delay. 

We have investigated various concatenation schemes 
with Turbo codes in an attempt to achieve some or all 
of the following: 

-Eliminate the error floor. 

-Achieve similar performance with less decoding effort 
(fewer iterations of decoding). 

-Achieve similar performance with smaller interleaver 
size (reduced delay). 



Lowering the Error Floor 


The presence of an error floor is due to the Turbo 
codes’ small free distance. A small number of errors, 
usually a small even number of errors in a block, can 
often ‘fall through’ the iterative decoding process and 
not be corrected. 

Concatenating a Turbo code with an outer code, 
which is optimized for free distance, can lower the 
error floor. The outer code will pick up the residual 
small number of errors and correct them. 

A concatenation scheme with an inner Turbo code 
and an outer Reed-Solomon code: 










Estimated BER of this scheme: 




K - 1 


P b < 




2 K -lji? +l n 



p b —bit error rate at the RS decoder output 
p s —symbol error rate at the RS decoder input 
n —block length of RS code in symbols 

K —number of bits per symbol 
t —error correcting capability of RS code 

We assume a t = 8, K = 8, n- 255, i.e., a (255, 239) RS 
code, in the following. 


• Based on a simulation of Turbo codes (block 
size=255x8=2040), the bounds were calculated using 
the above formula and showed a large decrease in the 
error floor. 
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Improvement with an Intra-code Bit Interleaver 

• The error distribution of the decoded bits within a 
Turbo code block has some structure. The distance 
between two or more errors is not uniformly 
distributed. If there is an error at position k , then the 
probability of another error at position k + j • n cyde , 

where n de is the cycle length of the recursive con- 
volutional code, is high. 




• In the previous concatenation scheme, we can make 
use of this fact to decrease the symbol error rate at 
the input to the RS decoder: 



Output of Turbo-Decoder Deinterleaver Input for RS-Decoder 






m in 01 


• Bounds calculated with the intra-code bit interleaver 
show improvement(not much, though): 



Eb/No [dB] 




BER 


• Thus with an outer (255,239) Reed-Solomon code, the 
error floor can be lowered significantly while the 
overall code rate only decreases from 0.50 to 0.47. 





Further Discussion of These Results 

Formula (1), which was used to estimate the BER of 
the concatenation scheme, assumes that the symbol 
errors are random, i.e., ideal symbol interleaving 
between the inner and outer codes. However, 
extensive simulations show that this is not the case 
with Turbo codes. 

Assuming random symbol errors, and a symbol error 
rate = p s , then the probability that there are i symbol 
errors in a block is: 

CKa-pj 255 "' (2) 

and when p s is small, the probability that there are 
many symbol errors in a block is very small. 

On the other hand, we can calculate the relative 
frequency of i symbol errors in a block, R.F.(i), based 
on simulation results: 

„ „ „ # of blocks with i symbol errors 

R.F.(i) = — ( 3 ) 

total # of blocks 




• Clearly the symbol errors at the input of the RS 
decoder are not randomly distributed. Take a close 
look at the simulation results: 
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•For Turbo codes, most blocks are error free. There 
are a lot of blocks with 2 symbol errors(caused by 2 
bit errors). There are a few blocks with many symbol 
errors, which is almost impossible when assuming 
random symbol errors. 



In the above example, the blocks which have more 
than 8 symbol errors cannot be corrected by the outer 
RS code, which means the BER bounds obtained 
using formula (1) are better than the actual 
performance. 

Bit interleaving can decrease the symbol error rate at 
the input to the RS decoder, but it doesn’t help much 
for the blocks with many symbol errors. 



# of blocks = 3000, SNR=1.21 dB, 5 iterations 
Without bit interleaving: 
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# of blocks with more than 8 symbol 

errors = 43 






With bit interleaving: 
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# of blocks with more than 8 symbol errors = 40 



Using Symbol Interleaving 

• Symbol interleaving can be used to spread the many 
symbol errors in one block to several blocks, thus 
making the symbol errors more random and 
generating more correctable blocks. 


Interleaving depth = 5 

































# of blocks=21000, SNR=1.21 dB, 5 iterations 


Interleaving depth = 0 
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# of blocks with more than 8 symbol errors = 256 


Interleaving depth = 5 
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Interleaving depth = 15 


0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15042 3596 1146 484 

273 

169 

123 

82 

36 

23 

15 

6 

1 

1 

1 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

0 

2 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


# of blocks with more than 8 symbol errors = 49 



# of blocks = 21000, SNR =1.21 dB, 15 iterations 
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# of blocks with 

more than 8 svmbol errors = 6 







Interleaving depth = 
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Interleaving depth = 

15 
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For 5 iterations of decoding, symbol interleaving gives 
significant improvement. However, for 15 iterations 
of decoding, interleaving doesn’t give much improve- 
ment even when the interleaving depth is 15. 

Increasing the interleaving depth makes the symbol 
errors more random, but at the cost of a longer 
overall delay. 

We are currently investigating combining bit inter- 
leaving with symbol interleaving to give further 
improvement. 



• We can use the simulation results to estimate the 
actual performance: 


P b < 


128 

255 


f*j 7 + 8 
£,N 255 



where 

n j is the # of blocks with j symbol errors 
N is the total # of blocks 


For the results to be accurate, N must be large. 


• Example: at SNR=1.21 dB, 5 iterations, interleaving 
depth =15, 


Pb< 


128 23x17 + 15x18 + 6x19 + 1x20 + 1x21 + 1x23 + 2x24 
255 21000 x 255 


= 8xl0' 5 


compared to p b = 4 x l(T 9 assuming ideal interleaving 
( see previous curve ). 



• When using symbol interleaving, the overall delay 
increases with interleaving depth. On the other hand, 
the performance of Turbo codes improves with the 
block size. As far as the delay is concerned, (symbol) 
interleaving to depth 5 is equivalent to one Turbo 
code block (block size=10200) containing 5 RS code 
words. The scheme with a larger Turbo code block is 
much better than the scheme with symbol inter- 
leaving. 


# of RS blocks = 21000 (# of Turbo blocks = 4200), SNR =1.21 dB, 15 iterations 
(no symbol interleaving between the RS codes in one Turbo block) 

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 

20903 14 78 131000000 000 


# of RS blocks with more than 8 symbol errors = 0 



Using Feedback from the Outer Code 


• The performance of the concatenated scheme can be 
further improved by using feedback from the RS 
decoder to the Turbo decoder. With the help of 
feedback from the outer code, we can move the 
performance curve closer to capacity or get similar 
performance with fewer Turbo decoding iterations. 

• Different rate RS codes can be used to provide 
feedback. After some iterations of Turbo decoding, 
the stronger RS codes are decoded and those bits are 
fed back to assist subsequent iterations of Turbo 
decoding; after some further iterations of Turbo 
decoding, the same process is repeated for the weaker 
RS codes. 

• There are two different ways to use the feedback 
information in Turbo decoding: 

--The initialization of the extrinsic information L is 
changed. Normally this value is 0 before the first 
iteration, but based on the RS decoder output, the 
extrinsic information can be set to a predetermined 
value or changed by a predetermined amount. 

—The channel metric of the information sequence is 
changed based on the RS decoder output. 



To see the different effect of the two methods, we 
assume that 10% of the information bits are fed back 
with a certain reliability (rate 1/2, block size 2040, 
Turbo code). The extrinsic information is initialized 
for the ‘known’ information bits with 

L = ±TC inU (5) 

or the channel metric of the information sequence is 
changed as follows: 


y;=y;±TC M „ (6) 

where ± depends on the estimated bits from the RS 
decoder. 















• The improvement achieved by changing the initia- 
lization values of the extrinsic information is 
negligible because L tends to grow to large values 
during the iterative decoding process for most of the 
information bits and a change in the initial value of 
the extrinsic information has its influence only for the 
first few iterations. 

• A modification of the metrics of the received 
information sequence, however, has a significant 
effect on the decoding performance because a change 
in the information sequence maintains its influence 
during each decoding iteration. The optimal value of 
TC init is a function of the channel SNR and the 

reliability of RS decoding. To give positive results, the 
feedback bits must have a high reliability. 

• Hence we are focusing on feedback through a change 
in the channel metric. Two schemes are proposed: 

—Only a small fraction of the information bits in a 
block are encoded using one low rate RS code, or 
several codes with different rates, interleaved into the 
information sequence prior to Turbo encoding. 

—All the information bits in a block are encoded using 
one high rate RS code, or several codes with different 
rates, interleaved, and then Turbo encoded. 



Difference between the two concatenation schemes: 

-For the first scheme, the rate loss can be very small. 
Its main purpose is to reduce the number of iterations 
of Turbo decoding, although some lowering of the 
error floor may also result. This scheme can be quite 
natural in applications where some bits in a block 
need extra protection. 

-For the second scheme, the number of iterations of 
Turbo decoding can also be reduced. Furthermore, 
after Turbo decoding, the outer codes can pick up 
most of the residual errors and lower the error floor 
significantly. But the rate loss is higher. 



When to Feed Back 

-If the RS code is decoded correctly, better per- 
formance can be achieved if the decoded bits are fed 
back earlier and with a higher confidence level; if 
there are errors in the RS decoding, the performance 
may be worse than without feedback. Therefore, it is 
not wise to perform the feedback until after a fixed 
number of Turbo decoding iterations. 

-For the commonly used Berlekamp-Massey algorithm 
for RS decoding, when the degree of the error 
location polynomial is greater than t or the number of 
roots is not equal to the degree, errors are detected, 
and the probability of undetected error is very low. 
This fact can be used to determine when to begin the 
feedback. 



Preliminary results show that, for the first scheme, 
with a block size of around 65,000 , at a channel SNR 
=0.7 dB, a factor of 4 improvement in BER was 
achieved with a rate loss of only 3%. 

We are currently investigating the performance of 
these schemes in an attempt to find the best 
concatenation strategy. 

An outer BCH code can also be used to reduce 
average decoding complexity. With an outer 32 bit 
CRC, we can decrease the number of iterations. More 
than half of the blocks in a Turbo code with an 
interleaver size of 20400 bits can be decoded correctly 
with fewer than 4 iterations at an SNR= 0.7 dB. 



Part IV 


Serial Concatenated 
Convolutional Codes 



Turbo codes show that good performance can be 
achieved by using a parallel concatenation of 
convolutional codes, but it is also possible to get good 
performance using serial concatenation. 

Example of a rate 1/2 serial concatenated code: 



Encoder 1 is a rate 2/3 convolutional code, encoder 2 
is a rate 3/4 convolutional code. The serial concatena- 

2 3 1 

ted code has an overall rate of R = = - . 

3 4 2 



An iterative decoding process, which is similar to 
Turbo decoding, is proposed to decode the serial con- 
catenated convolutional codes. 


sw 



Two decoding algorithms, one based on interleaving 
bits and the other based on interleaving symbols, 
have been tried. 
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Comparison of different decoding algorithms for serial concatenated codes 

(Block size: 400 bits. Rate: 1/2) 




A comparison between the two decoding algorithms 
shows that: the symbol decisions algorithm is better 
for low SNR’s; the bit decisions algorithm is better 
for high SNR’s, 

Different component codes (that is, feedforward and 
feedback convolutional codes) were tried for the inner 
and outer codes. Results show that using a feedback 
inner code with a feedforward outer code gives better 
performance. 

Recently new progress with serial concatenation has 
been reported by Benedetto and Divsalar, which 
shows that this scheme can offer superior per- 
formance to Turbo codes, and code design rules are 
given. 



Topics for Further Research 


Consider the effect of interleaving only some of the 
sequences coming from the outer code instead of all of 
them. 

Code design rules for low SNR’s. 

More than two (e.g., 3) component codes in the serial 
concatenation scheme. 

A thorough comparison of parallel concatenated 
convolutional codes and serial concatenated convo- 
lutional codes. 




